The present invention relates to display methods and apparatus and, more particularly, to methods and apparatus for displaying images, such as text on display devices which have multiple separately controllable luminance elements per pixel.
The display of images, e.g., text characters, on display devices is of high importance. This is particularly the case given the ever increasing use of computers and other types of devices which rely on displays to convey information.
Pixels are used to represent display images on a display device. The term pixel, which is short for picture-element, is commonly used to refer to one spot in, e.g., a rectangular grid of thousands of spots which can be used to represent an image. Pixels are normally used individually by a computer to form an image on the display device.
Color display devices have become the principal display devices of choice for most computer users. The display of color on a monitor is normally achieved by operating the display device to emit light, e.g., a combination of red, green, and blue light, which results in one or more colors being perceived by a human viewer.
In color displays, the intensity of the light emitted corresponding to the additive primary colors, red, green and blue, can be varied to get the appearance of almost any desired color pixel. Adding no color, i.e., emitting no light, produces a black pixel. Adding 100 percent of all three colors results in white.
In cathode ray tube (CRT) display devices, the different colors of light are generated via the use of phosphor coatings which may be applied as dots in a sequence on the screen of the CRT.
Liquid crystal displays (LCDs), or other flat panel display devices are commonly used in portable computer devices in the place of CRTs. This is because flat panel displays tend to be small and lightweight in comparison to CRT displays. In addition, flat panel displays tend to consume less power than comparable sized CRT displays making them better suited for battery powered applications than CRT displays.
As the quality of flat panel color displays continues to increase and their cost decreases, flat panel displays are beginning to replace CRT displays in desktop applications. Accordingly, flat panel displays, and LCDs in particular, are becoming ever more common.
Unlike CRT displays, LCD displays use square or rectangular light emitters, usually positioned adjacent one another, as the source of red, green and blue light for each pixel. Due to manufacturing limitations with regard to the size of light emitters in LCD displays, it is difficult in an LCD display to achieve the same resolution commonly found in CRT displays.
Unfortunately, the limited resolution provided by commonly used flat panel displays such as LCDs tends to be less than ideal for the display of text. LCD display resolution problems are particularly noticeable when displaying text at small font sizes which are commonly used on personal data assistants and other hand held devices. When the size of a pixel is treated as the smallest unit of size which can be used to represent a position of a character or other image, the relatively large pixel size compared to the character size tends to produce characters with jagged edges.
The use of LCD displays with small characters can also produce less than desirable representations of, e.g., bold text. In the case of bold text it is desirable that bold character stems have stem weights that are 10-20% higher than the corresponding non-bold character stem. At small sizes a character stem may be only one or two pixels in width. Given such stem widths, adjustments in stem weights in one pixel size increments as is commonly done in the art, can lead to far more than the desired 10-20 percent increase in stem weight for bold characters.
FIG. 1 illustrates a known computer system 100 which comprises a housing 101, keyboard 104, disk drive 105 and an LCD display 102. The LCD display comprises a plurality of pixels, two of which are identified by reference numbers 110, 112. Each of the pixels 110, 112 includes separate red (R), green (G) and blue (B) pixel subcomponents which may be controlled as independent luminous intensity sources. In the computer system 100, the R, G and B pixel subcomponents are arranged to for vertical stripes.
In known systems, in the case of text, a high resolution representation of a text character, sometimes referred to as an analytic representation, is normally sampled. The samples are then used to generate luminous intensity values, e.g., red, green and blue pixel sub-component luminous intensity values, which control the light output of the R, G and B pixel sub-components of each pixel, respectively. In conventional systems, the R, G, and B pixel sub-component luminous intensity values are normally generated from the same set of image samples.
FIG. 2 illustrates one known image rendering technique used to control the light output of a set of pixels. In FIG. 2, the grid 220 represents a source image, e.g., a foreground/background color representation of a text character, which has been divided into 9 segments corresponding to 3 rows R(N), R(N+1), R(N+2) and 3 columns C(N), C(N+1), C(N+2) of source image pixel segments. Each one of the 9 segments corresponds directly to one pixel, e.g., pixel 240, of the display screen represented by grid 230. Red, green and blue pixel sub-components 232, 234, 236 are illustrated in FIG. 2 using medium, dark and light speckling, respectively.
In the known image rendering technique, each portion of a source image corresponding to a pixel, referred to herein as a source image pixel segment, is oversampled by a factor of 4 in the vertical and horizontal dimensions. Thus, a set of 16 samples, represented by the circles 222 with x""s inside, is generated for each pixel.
The 16 samples corresponding to each pixel segment are then filtered to produce the red, green and blue luminous intensity values used to control pixel sub-components 232, 234, 236. The filtering of the samples is represented by the arrow extending from source image segment 223 to pixel 240. Thus, in the illustrated system, the same portion of the source image is used to generate each of the red, green and blue pixel sub-component luminous intensity values of a pixel. In the known FIG. 2 system, the filtering performed to generate pixel sub-component luminous intensity values does not cross pixel boundaries indicated in the image 220 by the use of solid lines. Accordingly, the luminous intensity of each pixel is not affected by neighboring source image pixel segments. As will be discussed below, this allows different images, e.g., text characters, to be sampled, filtered, stored and then subsequently concatenated without impacting the filtering since the filtering does not depend on neighboring image portions beyond a pixel boundary.
As is known in the art, there are generally two stages associated with the display of text images, e.g., characters, 1) the glyph rendering stage and 2) the glyph display phase. The glyph rendering stage involves the generation of one or more character glyphs and the storage of the glyphs in a glyph cache for subsequent use, e.g., during the display phase. The glyph display phase involves retrieving glyphs from the font cache as need, and, in many cases, combining them prior to display to form text strings.
In the glyph rendering stage one or more character glyphs are rendered, i.e., generated, from corresponding high resolution representations, e.g., outline representations, of the rendered characters. The high resolution character representations from which characters are frequently rendered include character shape and size information. The shape information is frequently in the form of lines, points, curves and/or arcs. Areas within the character outline correspond to the foreground color while areas outside the character outline correspond to the background color.
As part of one known character glyph rendering process, the high resolution representation of the character image from which the glyph is being generated, is sampled multiple times per pixel in both the horizontal and vertical directions as illustrated in FIG. 2. The samples corresponding to each pixel of an image are then filtered, e.g., summed, to perform a filtering operation. The value associated with each pixel resulting from the filtering operation is then stored, along with character spacing information, in a font glyph cache. In the known system one value, referred to as an alpha value, is generated per pixel as a result of the filtering operation.
Alpha values indicate the proportion of the foreground and the background color which contribute to the image area to which the alpha value corresponds. In the known system a single alpha value is generated for each pixel and is used to control the red, green and blue luminous intensity values of the pixel as a function of foreground and background colors which are subsequently applied.
Alpha values are usually normalized to a value in the range of 0-1 before being used in the application of foreground/background colors to a rendered character glyph. In the known system a normalized alpha value of 1 indicates that the region to which the alpha value corresponds is a foreground color region. A normalized alpha value of 0.5 indicates that the foreground and background colors contribute equally to the color of the region corresponding to the alpha value. In addition, a normalized alpha value of 0 indicates that the region to which the alpha value corresponds is a background color region.
In the known system, rendered glyphs, which include character spacing information and one alpha value per pixel, are stored in a glyph cache. The stored alpha values are non-normalized values which are generated by summing the individual samples, having a value of 0 or 1, corresponding to a pixel region.
By storing rendered glyphs in a glyph cache, the known system avoids the need to render a character glyph each time it needs to be displayed, e.g., as part of a text string.
The second phase of rendering text, the glyph display phase, involves reading out glyphs from the glyph cache, applying foreground/background color selections, a gamma correction operation and then outputting the resulting values to a display buffer for storage until being used to control the display device to display the desired character images.
The above described process can be used to generate and display opaque or transparent characters. In the case of opaque characters, a uniform background color is applied. In the case of transparent characters, the background color may vary as a function of the background image over which the characters are placed.
FIG. 3 illustrates one known embodiment of the above described glyph rendering and displaying process as applied to rendering and displaying opaque characters. In FIG. 3, and the various other process diagrams included in the present application, ovals are used to illustrate steps, rectangles are used to illustrate hardware devices, e.g., storage devices, and non-rectangular parallelograms are used to represent data, information, or other values.
The known process 300 begins in step 302 with the sampling of source image pixel segments of a character image. The pixel segments are sampled at a rate of 4xc3x97 in both the horizontal and vertical directions as illustrated in FIG. 2. The image samples (16 per pixel) 304 are then filtered in step 306 using a box filter to produce a value between 0 and 16 per pixel. Next in step 308, values of 16 are clipped to 15 so that the alpha value for each pixel can be represented using 4 bits. This permits the alpha values to range between 0-15 and allows the alpha values to be stored using 4 bits each as opposed to 5 bits which would be needed for the range 0-16. The alpha values produced by clipping step 308, on per pixel, are stored in the font glyph cache 310 as a glyph corresponding to the sampled image. Multiple character glyphs may be stored in the font glyph cache 310.
The glyph display phase of display text begins with glyphs being output by the glyph cache 310. The output of glyphs is in response to a glyph identifier which identifies the stored glyph to be output.
The alpha values, one per pixel, included in the output glyphs are processed to apply selected foreground and background colors in blending step 312. As part of step 312, the alpha values are normalized to a value between 0 and 1. Foreground and background colors are then applied, to generate R, G, and B luminous intensity values, on a per pixel according to the following equation:
R=xe2x88x9dRF+(1xe2x88x92xe2x88x9d)RB
G=xe2x88x9dGF+(1xe2x88x92xe2x88x9d)GB
B=xe2x88x9dBF+(1xe2x88x92xe2x88x9d)BB
Note that foreground and background colors are specified as gamma corrected values, i.e., values which have been processed to take into consideration the non-linear response characteristics of a display device. Before use in the color blending operation 312, the gamma corrected foreground and background colors are inverse gamma corrected in steps 311 and 312 to produce the foreground and background luminous intensity values used in the color blending process.
The R, G, B foreground and background luminous intensity values used for blending are indicated through the use of an F subscript for a foreground value and a B subscript for a background value.
The generated luminous intensity values are gamma corrected in step 314 prior to being stored in the display buffer 316. Gamma correction is a well known process for adjusting luminous intensity values to take into consideration the non-linear nature of the light output of a display device in response to voltage control signals.
FIG. 4 illustrates another embodiment of the known process for rendering and displaying opaque glyphs. In the FIG. 4 process 400, a pre-computed look-up table 402 is used to perform a combined color application and gamma correction procedure. A different look-up table 402 is used for each supported set of foreground/background colors. The look-up table 402 receives as its input one four bit alpha value per pixel and outputs one set of R, G, B values for each alpha value supplied thereto.
FIG. 5 illustrates a known process 500 for rendering and displaying transparent glyphs. In the FIG. 5 embodiment, gamma correction is performed in the alpha color space in step 502 on the alpha values output by the font glyph cache 310. The gamma corrected alpha values are then supplied to compositing step 504. Since gamma correction is performed in alpha space inverse gamma correction operations need not be performed on the foreground/background color values prior to color blending. Compositing step 504 is responsible for performing color blending using a fixed foreground color and a background color obtained from a background image. Accordingly, in the process 500, background image data is retrieved, on a per pixel basis, from the display buffer 516 in compositing step 504 for color blending purposes. The background information is retrieved from the pixel location in the display buffer 516 corresponding to the image location of the pixel being prepared for display. The R, G, B luminance values produced for each pixel are stored in the display buffer 516 at locations indicated by the glyph location information supplied to compositing step 504.
While the known text display methods work satisfactorily in many applications, as discussed above, in the case of LCD displays and small display sizes, images can suffer from blur, jaggedness, and other problems due to a lack of resolution. This is due, in part, to the fact that pixels are treated as single entities in most known image display processes.
In view of the above remarks, it is apparent that there is a need for new and improved display methods and apparatus. It is desirable that at least some of the new methods and apparatus be capable of displaying small size text on LCDs at reasonable quality levels. It is also desirable that some of the new methods and apparatus be capable of displaying text in a computationally efficient manner allowing for the use of lower cost processors than might otherwise be required.
The present invention relates to methods and apparatus for displaying images, e.g., text characters, on display devices which have multiple separately controllable luminance elements per pixel.
In accordance with the present invention, pixel sub-components are treated as individual luminous intensity sources which can be separately controlled to represent different portions of a source image. As part of the display process of the present invention, high resolution representations of images, e.g., outline representations of text characters to be displayed, are super-sampled. Sets of samples corresponding to different but potentially overlapping image portions are then used to control the light output of individual pixel sub-components of each pixel. The samples, in the set of samples used to control each pixel sub-component are filtered, e.g., summed, to generate a filtered alpha value corresponding to a pixel sub-component. In this manner, a set of filtered alpha values, one for each pixel sub-component of a pixel is generated.
Foreground and background color is applied to each pixel sub-component through the use of alpha blending techniques.
The filtering step uses, in various implemented embodiments, alpha values from adjacent source image pixel segments. This can complicate the filtering process since the content of a neighboring pixel can be dependent at glyph boundaries on the content of a neighboring character glyph.
In one embodiment, non-filtered alpha values are stored in a glyph cache. When character stings are to be generated, the cached glyph data is output from the cache and then filtered. In such an embodiment, the filtering of the alpha values occurs after the glyphs are combined. Accordingly, in such an embodiment, alpha values from neighboring glyphs are available at filtering time. Thus, in the post cache filtering embodiment, color leakage across pixel boundaries does not present a problem.
In other embodiments, the alpha values used to represent a character glyph are filtered prior to the glyph being stored in the glyph cache. In such embodiments, the alpha values from neighboring glyphs are not available at the time of filtering since the neighboring glyph remains undetermined. In such an embodiment, it is not possible to take into consideration the effect of neighboring glyphs on the glyph being filtered at the time of filtering.
To address the effect of neighboring glyphs on filter output values, in one embodiment, glyphs are padded prior to filtering to add one or more background color pixel sub-components at the points along the glyph vertical edges where color leakage might occur from the character glyph as a result of filtering.
Subsequent to filtering, the padded glyphs are stored in the glyph cache. In such a pre-cache filtering embodiment, when padded glyphs are output from the glyph cache and combined to represent a text string, filtered alpha values corresponding to overlapping image portions are summed. The resulting filtered alpha values are identical to those which would have resulted if the pre-filtering alpha values of adjacent character glyphs were combined and then filtered.
The pre-glyph cache filtering embodiment allows glyphs to be filtered once, e.g., prior to storage in the glyph cache with a minimal amount of processing, e.g., summing of alpha values, then being required to patch the edges of glyphs which are combined to form larger images. Thus, the repeated filtering of the entire contents of a glyph each time the glyph is output from the glyph cache, is avoided with the pre-cache glyph filtering embodiment of the present invention.
Methods and apparatus for efficiently darkening characters which may otherwise appear too light or gray, due to thin stem widths, are also described. The described methods involve bloating character images of particular fonts with which the gray problem due to thin stems is associated. The bloating operation may be selectively implemented as part of a filtering look-up table operation. Such an approach to resolving the gray character problem avoids the need to modify hinting rules or modify the character font file from which the character images are generated.
Additional features, embodiments and benefits of the methods and apparatus of the present invention are discussed in the detailed description which follows.